Skip to content

add --num-shards option for milvus performance test case#526

Merged
alwayslove2013 merged 2 commits intozilliztech:mainfrom
LoveYou3000:feature/add-num-shards-for-milvus
May 20, 2025
Merged

add --num-shards option for milvus performance test case#526
alwayslove2013 merged 2 commits intozilliztech:mainfrom
LoveYou3000:feature/add-num-shards-for-milvus

Conversation

@LoveYou3000
Copy link
Copy Markdown
Contributor

No description provided.

@alwayslove2013
Copy link
Copy Markdown
Contributor

@LoveYou3000, thanks for your PR, but I'm curious why you wanna change the num_shards for Milvus. Given that each shard in Milvus can handle at least 200 million data rows and that the public datasets provided by vdbbench are significantly smaller, we did not include an option for num_shards when designing the Milvus client.

Are you testing with a much larger dataset, or do you have a specific need for extremely high write speeds? Otherwise, we generally do not recommend increasing the number of shards.

@LoveYou3000
Copy link
Copy Markdown
Contributor Author

We are doing a search performance test for the milvus cluster and want to compare the impact of different num_shards on the retrieval QPS.
BTW, my dataset is openai-large-10M

@sre-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: alwayslove2013, LoveYou3000
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@alwayslove2013 alwayslove2013 merged commit 571de32 into zilliztech:main May 20, 2025
4 checks passed
@alwayslove2013
Copy link
Copy Markdown
Contributor

@LoveYou3000 Thanks for your willingness to test Milvus more comprehensively! We would love to hear your testing results, as they could be very valuable to the community.

Just a gentle reminder: considering the size of your test dataset, a single shard should be sufficient to handle it effectively.

@LoveYou3000 LoveYou3000 deleted the feature/add-num-shards-for-milvus branch May 27, 2025 02:50
euphoria0-0 pushed a commit to CryptoLabInc/VectorDBBench that referenced this pull request Nov 21, 2025
)

* add --num-shards option for milvus performance test case

* fix lint warning
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants